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I. Introduction 


Introduction to the Handbook 

This Handbook provides an introduction to basic procedures and methods of data 
analysis. We provide a framework to guide program staff in their thinking about these 
procedures and methods and their relevant applications in MSHS settings. Accordingly, 
this Handbook was developed to support the work of MSHS staff across content areas. 

This Handbook was created because “using data in meaningful ways” includes many 
possibilities - and a great deal of information. Some of this information is complex and 
highly dependent upon specialized knowledge. However, there are many ways of “using 
data” that are more basic and that involve relatively simple procedures. Our purpose is to 
provide MSHS programs with a basic framework for thinking about, working with, and 
ultimately benefiting from an increased ability to use data for program purposes. 

Section I of the Handbook provides introductory information. Section I includes the 
presentation of the “guiding principles” we use in presenting information, as well as the 
articulation of specific learning objectives. In Section II, we present an introduction to 
ways of thinking about data and make the distinction between two basic types of data: 
quantitative and qualitative. Section III outlines information that we hope will assist 
program staff understand how to conceptualize data collection and its analysis as a 
process. In Section IV we discuss management considerations that should be addressed 
in order to gather and analysis data in an efficient and effective way in your program. We 
present a framework for managing the process of data collection and analysis. Because 
using data for program purposes is a complex undertaking it calls for a process that is 
both systematic and organized over time. 

In Section V of the Handbook we examine data analysis using examples of data from 
each of the Head Start content areas. We explore examples of how data analysis could be 
done. We identify and describe trends in data that programs collect. Finally, we offer a 
perspective of how data lends itself to different levels of analysis: for example, grantee- 
wide, by delegate agency, and/or center- or classroom-level. Our intent is to demonstrate 
how the different analytical procedures and methods can be powerful and effective tools 
for MSHS managers. Specific procedures and methods of data analysis are discussed with 
clear ways of using and working with data in order to identify results. Here, procedures 
and methods for working with both quantitative and qualitative data are presented. 
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Learning Objectives 

This Handbook is used in training sessions offered by the Migrant and Seasonal Head 
Start Technical Assistance Center (TAC- 12). It can also be used as a self-paced or group 
resource. Utilizing this Handbook, you will explore how to: 

• use a wide variety of data for planning and decision-making purposes; 

• begin to develop abilities to use data to describe program operations and/or practices; 

• observe basic techniques of data analysis to real-life Head Start examples; and 

• identify and articulate trends and patterns in data gathered over time. 

Guiding Principles for Approaching Data Analysis 

1. To provide information to program staff from a variety of different backgrounds and levels of 
prior experience. 

2. To create a “value-added” framework that presents strategies, concepts, procedures, methods 
and techniques in the context of real-life examples. 

3. To appreciate that learning takes time. 

4. Comfort, confidence, and competence take practice. 

5. Data analysis provides opportunities to “reduce the burden.” 


What This Handbook Does NOT Do 

This Handbook does not provide guidance on data collection. For those who are familiar 
with the Five Steps to Community Assessment: A Model for Migrant and Seasonal Head 
Start programs workbook, that resource can provide you with good strategies for data 
collection. 

In addition, this Handbook does NOT attempt to address all possible procedures or 
methods of data analysis or imply that “data analysis” is limited to the contents of this 
Handbook. Program staff are urged to view this Handbook as a beginning resource, and 
to supplement their knowledge of data analysis procedures and methods over time as part 
of their on-going professional development. In addition, fiscal analysis procedures such as 
analyzing cost per child per hour, etc are beyond the scope of this handbook. 
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This Handbook does not provide information on how to complete the Program 
Information Report (PIR). This information is provided by the PIR contractor, Xtria, in 
the form of an annual User’s Guide (see www.xtria.com/pir2005). Programs who require 
assistance in completing the PIR are instructed to contact their designated Program 
Specialist. 

This Handbook does not teach how to use Excel or other spreadsheet programs. Training 
on data entry, use of spreadsheets, and other applications is available from a variety of 
sources. Suggestions are included in Appendix B. 

Finally, we cannot answer questions about how Head Start program monitoring will 
be conducted in the future, including which types of data will be gathered and used in 
program reviews. 


Why Do We Need Data Analysis? 

Data is short hand for “information,” and whether you are collecting, reviewing, and / 
or analyzing data this process has always been part of Head Start program operations. 
Children’s enrollment into the program requires many pieces of information. The 
provision of health and dental services includes information from screening and any 
follow-up services that are provided. All areas of a Head Start program - content and 
management - involve the collection and use of substantial amounts of information. For 
MSHS programs, the use of data becomes even more crucial, as essential information 
must be managed within relatively short program seasons. Very few people working in 
MSHS programs complain about not having enough to do! 

In addition, Head Start program operations have evolved substantially over the last ten 
years. As new requirements have been added, and new program initiatives launched, 
programs are increasingly expected to use data in meaningful ways. Finally, if the PRISM 
process will “focus on the collection, reporting, and analysis of data,” programs are well- 
advised to develop their expertise in working with data. 

Lastly, recent Head Start sources support the need for a better understanding of data 
analysis in our work. 
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From the Head Start Bureau website (October 4, 2005): 

The Head Start Bureau has made some significant changes to procedures guiding the 
FY 2006 PRISM. The changes will affect areas such as on-site review, the relationship 
between Federal team leaders and home region, and the way grantee and delegate agency 
reviews are conducted. The new changes to be implemented will also impact the PRISM 
Instrument procedures for reviewing service areas. An increased emphasis will focus on 
the collection, reporting, and analysis of data. 

From Information Memorandum ACYF-IM-HS-05-08 (10/04/05): 

Changes to the PRISM Process 

The following changes to the PRISM process are in place for fiscal year 2006: 

The on-site review will focus on the collection, reporting, and analysis of data. The 
grantee presentation and summary meeting will no longer occur. 

Federal team leaders will not supervise reviews in their home region when conducting 
triennial and first year reviews. These team leaders will, however, continue to supervise 
follow-up reviews for grantees in the Federal team leader’s home region. 

For grantees with delegate agencies, each delegate agency will be reviewed concurrently 
with the grantee agency review as part of each triennial and first-year review. 

http://www.headstartinfo.org/publications/im05/im05_08.htm 
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II. Ways of Thinking About Data 


Data is... 

The 1973 Webster’s New Collegiate Dictionary defines data as “factual information (as 
measurements or statistics) used as a basis for reasoning, discussion, or calculation.” The 
1996 Webster’s II New Riverside Dictionary Revised Edition defines data as “information, 
especially information organized for analysis.” Merriam Webster Online Dictionary 
defines data” as the following (http://www.m-w.com): 

1 : factual information (as measurements or statistics) used as a basis for reasoning, 
discussion, or calculation. E.g., the data is plentiful and easily available -- H. A. 
Gleason, Jr., e.g., comprehensive data on economic growth have been published 

-- N. H. Jacoby. 

2 : information output by a sensing device or organ that includes both useful and 
irrelevant or redundant information and must be processed to be meaningful. 

3 : information in numerical form that can be digitally transmitted or processed. 

Taking from the above definitions, a practical approach to defining data is that data 
is numbers, characters, images, or other method of recording, in a form which can be 
assessed to make a determination or decision about a specific action. Many believe 
that data on its own has no meaning, only when interpreted does it take on meaning 
and become information. By closely examining data we can find patterns to perceive 
information, and then information can be used to enhance knowledge (The Free On-line 
Dictionary of Computing, 1993-2005 Denis Elowe). 


• The number 1,099 is one example of data. 

• “The number of children who were determined to have a disability 
prior to enrollment in Migrant and Seasonal Head Start for the 2004 
enrollment year is 1,099” is information. 


© AED/Tac-12 Spring 2006. Permission granted to copy for non-commercial uses only. 


5 




What has been evident in disciplines such as education, public health, nutrition, nursing, 
and management, is now becoming evident in early care and education, including Head 
Start. Programs now recognize that the quality and quantity of data, be it statistical or 
descriptive, is needed to set baselines, identify effective actions, set goals and targets, 
monitor progress and evaluate impacts (World Bank Website http://www.worldbank.org/ 
data/ aboutdata/ aboutdata.html) . 

One thing that Migrant and Seasonal Head Start programs can do well is gather data. 
Using an upstream program as an example, in late May the data gathering process is swift 
and fast during enrollment. If the data relates to a child or family it is shared among 
appropriate Head Start staff. When the program ends, the data is stored, and next thing 
you know it is March and the program is preparing for pre-service. The question that 
one is always left with is: what do we do with all this data and or information? One of the 
goals of this handbook is to help you answer this question. 

Before you can present and interpret information, there must be a process for gathering 
and sorting data. Once again, 1,099 is a number - and this number is, in fact, data. The 
number 1,099 is a raw number - on its own it has no meaning. Just like many of the crops 
that our families pick are raw from which food is prepared, so too, can data be viewed as 
the raw material from which information is obtained. 

Head Start requires the collection of data in a variety of areas. We collect data in all 
of the content service areas. Thus, data collection is something that is not just limited 
to children and families, but if the purpose and the questions relate to children and 
families it is definitely not good practice to collect data when the children and families 
are not available. The data you collect in Head Start can take many forms. The data 
could be in the form of numbers, words, pictures, maps, and even newspaper articles. 
When collecting data, we are faced with the inevitable question of which is better. The 
concept of which is better has the potential to lead to the qualitative versus quantitative 
debate, which although exhilarating to some, could cause havoc in program planning and 
implementation. These debates fail to achieve an honest understanding of how qualitative 
and quantitative data differ, because in many people’s mind the difference between the 
two is underscored by the notion that one is better than the other. 

Why the Soliloquy? Types of Data 

In research circles there has been a long-term debate over the merits of Quantitative 
versus Qualitative data. Key influences in this debate are based upon how researchers 
were taught, compounded by differences among individuals and their preference in 
relating to numbers or to words. 

In reality, this debate is largely irrelevant in Head Start. In order to have a high quality 
program, we must collect both types of data. There are times when a quantitative 
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approach will be better suited to the situation and vice versa. “Qualitative and quantitative 
methods are not simply different ways of doing the same thing. Instead, they have 
different strengths and logics and are often best used to address different questions and 
purposes (Maxwell, 1996,2005).” That being said, there are other times when it makes 
sense to “have the best of both worlds,” and to use a combination of some quantitative 
and some qualitative data in order to credibly address a particular question and make well 
informed decisions. 

Qualitative data 

Data that is represented either in a verbal or narrative format is qualitative data. These 
types of data are collected through focus groups, interviews, opened ended questionnaire 
items, and other less structured situations. A simple way to look at qualitative data is 
to think of qualitative data in the form of words. Later on, we will explore how the 
transcript below can be used as a source of data. 

Sample Qualitative Data: Transcript from Parent Interview 

(Family One - husband) 

OK, well, me first, before anything, I came here because of the poverty, do you 
understand me ? And for a good quality of life, in my country not so much thinking 
about myself but about my smaller sisters so that they can have a better education 
considering that I did not have any, and another reason was in my country there are 
no good jobs, and very little work. They pay you very little and you never leave the 
poverty and another reason is that my father had two brothers that were already here 
and I thought that the more of us that are here the better that everybody could help 
each other out and it would be easier to get ahead with our younger sisters. 

(Family One - wife) 

The same to help my parents I came and I would send money and then my brother 
came and then the same. 

(Family One - husband) 

Because in her case, she or better yet sometimes us (in our situation) because we were 
the eldest, we were men but in her case the eldest are women, and the ones that were 
going to work, theoretically to help the parents the most were the youngest. 

(Family One-wife) 

My dad was already here when I came, he was here, my mother was in Mexico, my 
dad stayed here for some time and then he went to Mexico. 
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(Family One-husband) 

More than anything, my parents for example they are that type of people that support 
you no they never stop, for example your dreams do not become reality they always 
try that if you decide, you know who you are, and they always let us what we wanted 
better yet things that they showed us and that were good we would do them and they 
always tried to better us. 

Quantitative data 

Quantitative data is data that is expressed in numerical terms, in which the 
numeric values could be large or small. Numerical values may correspond to a 
specific category or label. 

Sample Quantitative Data from PIR 

Actual Enrollment by Child 


Ages of children served: 

a. Under 1 year: 3,843 

b. 1 Year old: 4,785 

c. 2 Years old: 6,341 

d. 3 Years old: 7,604 

e. 4 years old: 6,988 

f. 5 Years and older: 3,462 


Contrasting Types of Data in Head Start 


Qualitative Data 

Family Partnership Agreements 
Social Service logs 
Advisory group minutes 
Policy Council minutes 
Newspaper articles 

Quantitative 

PIR 

Child performance tracking 
Health data tracking systems 

Mixed Data 

Enrollment info 

Enrollment & transition records 
Surveys (i.e., Parent, Teacher/Staff, 
Farmer) 
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Data Strategies 

There are a variety of strategies for quantitative and qualitative analyses, many of which 
go well beyond the scope of an introductory Handbook. Different strategies provide 
data analysts with an organized approach to working with data; they enable the analyst 
to create a “logical sequence” for the use of different procedures. In the boxes below, 
we offer four examples of strategies for quantitative analysis that you may consider as 
you work with and develop your skills in data analysis as well as reasons why you may 
consider using the strategy. Some of these strategies are used in Section V when looking 
at particular content area data. 

Strategy: Visualizing the Data 

Involves: Creating a visual “picture” or graphic display of the data. 

Reason(s): a way to begin the analysis process; or as an aid to the reporting/ 

presentation of findings. 


Strategy: Exploratory Analyses 

Involves: Looking at data to identify or describe “what’s going on”? - creating an 
initial starting point (baseline) for future analysis. 

Reason(s): Like you have a choice? 


Strategy: Trend Analysis 

Involves: Looking at data collected at different periods of time. 
Reason(s): to identify and interpret (and, potentially, estimate) change. 


Strategy: Estimation 

Involves: Using actual data values to predict a future value. 

Reason(s): to combat boredom after you have mastered all the previous strategies. 
Also to answer PIR and Community Assessment items and tasks. 
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Visualizing Data 

Visualizing data is to literally create and then consider a visual display of data. Technically, 
it is not analysis, nor is it a substitute for analysis. However, visualizing data can be a 
useful starting point prior to the analysis of data. 

Consider, for example, someone who is interested in understanding Migrant and Seasonal 
Head Start from a national perspective. Specifically, someone might be interested in the 
differences in funded enrollment across all MSHS grantees. Looking at a random list of 
funded enrollmment numbers (PIR, 2004) gives us one perspective: 


Random List of Funded Enrollment 


677 

377 

266 

601 

2719 

830 

8623 

754 

480 

391 

171 

425 

6984 

470 

75 

1762 

259 

274 

217 

684 

2709 

70 

50 

133 

402 

2216 


In random order, it is a bit difficult to get a handle on the data. By ranking the values in 
order however (notes: this can be done either lowest to highest or highest to lowest) we 
gain a more organized perspective of the data set: 
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NUMBER OF CHILDREN (ENROLLMENT) 


Rank Order of Funded Enrollement 


50 

470 

70 

480 

75 

677 

133 

684 

171 

754 

217 

830 

259 

1601 

266 

1762 

274 

2216 

377 

2709 

391 

2719 

402 

6984 

425 

8623 


MSHS: Funded Enrollment by Grantee 2004 



GRANTEE 
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By creating a visual display of the data, we can begin to get a “feel” of how MSHS grantees 
differed in terms of their funded enrollment in 2004 using the numbers above (Note: In 
Excel, go to “Insert” and select “Chart” to convert a spreadsheet column into a bar chart, 
see Appendix B.). 

By creating and viewing a graphic display of the data, we get a “feel” of how MSHS 
grantees’ funded enrollment varies across the region. In particular, the size differences 
between the two largest grantees and the rest of the region stand out, as do the more 
basic differences between “small” and “large” programs. Again, this visual display of data 
is not a substitute for analysis, but it can often provide an effective foundation to guide 
subsequent analyses. 

Exploratory Analysis 

Exploratory analysis entails looking at data when there is a low level of knowledge about 
a particular indicator (teacher qualifications, first and second language acquisition, etc.) 

It could also include the relationship between indicators and/or what is the cause of a 
particular indicator. 

Trend Analysis 

The most general goal of trend analysis is to look at data over time. For example, to 
decern whether a given indicator such as the number of children with disabilities has 
increased or decreased over time, and if it has, how quickly or slowly the increase or 
decrease has occurred. One aspect of trend analysis that is discussed in this Handbook 
and encouraged is that of comparing one time period to another time period. This form 
of trend analysis is carried out in order to assess the level of an indicator before and after 
an event. 

Estimation 

Estimation procedures may occur when working with either quantitative or qualitative 
data. The use of both quantitative data such as poverty level data, can be combined with 
interviews from providers serving low income families to help determine the proportion 
of families in the area that are income eligible. Estimation is one of many tools used to 
assist planning for the future. Estimation works well for forecasting quantities that are 
closely related to demographic characteristics, eligible children and families, and social 
services. Estimation is the combination of information from different data sources to 
project information not available in any one source by itself. 
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III. Conceptualizing Data Analysis 
as a Process 


The “Problem” with Data Analysis 

What does ‘data analysis’ mean? Does it refer to one method or many? A collection of 
different procedures? Is it a process? If so, what does that mean? More important, can 
MSHS program staff - without a background in math or statistics - learn to identify and 
use data analysis in their work? (P.S. - the answer to the last question is Yes! - assuming a 
minimum investment of time, effort, and practice). 

Data analysis can refer to a variety of specific procedures and methods. However, before 
programs can effectively use these procedures and methods, we believe it is important to 
see data analysis as part of a process. By this, we mean that data analysis involves goals; 
relationships; decision making; and ideas, in addition to working with the actual data 
itself. Simply put, data analysis includes ways of working with information (data) to 
support the work, goals and plans of your program or agency. 

From this perspective, we present a data analysis process that includes the following key 
components: 

• Purpose 

• Questions 

• Data Collection 

• Data Analysis Procedures and Methods 

• Interpretation/Identification of Findings 

• Writing, Reporting, and Dissemination; and 

• Evaluation 

We have also found, from our review of the literature, that there are many different ways 
of conceptualizing the data analysis process. We can make a basic distinction between a 
linear approach and a cyclical approach; in this Handbook we provide examples of both. 
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Data Analysis as a Linear Process 

A strictly linear approach to data analysis is to work through the components in order, 
from beginning to end. A possible advantage of this approach is that it is structured and 
organized, as the steps of the process are arranged in a fixed order. In addition, this linear 
conceptualization of the process may make it easier to learn. A possible disadvantage is 
that the step-by-step nature of the decision making may obscure or limit the power of the 
analyses - in other words, the structured nature of the process limits its effectiveness. 


Data Analysis: In Linear Terms 
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Data Analysis as a Cycle 

A cyclical approach to data analysis provides much more flexibility to the nature of 
the decision making and also includes more and different kinds of decisions to be made. 

In this approach, different components of the process can be worked on at different 
times and in different sequences - as long as everything comes “together” at the end. A 
possible advantage of this approach is that program staff are not “bound” to work on 
each step in order. The potential exists for program staff to “learn by doing” and to make 
improvements to the process before it is completed. 



Therefore, the simplest possible answer to the question, what is data analysis, is probably: 
IT DEPENDS. Rather than chose to present ‘data analysis’ as either linear or cyclical, we 
have decided to present both approaches. Hopefully, this choice will give MSHS program 
staff the options and flexibility to make informed decisions, to utilize skills that they 
already have, and to grow and develop the ability to use data and its analysis to support 
program/agency purposes and goals. 
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Process Component #1. Purpose(s): 

What Do We Do? & Why? 

An effective data analysis process is based upon the nature and mission of the 
organization as well as upon the skills of the team that is charged with the task of 
collecting and using data for program purposes. Above all, an effective data analysis 
process is functional - i.e., it is useful and adds value to organizational services and 
individual practices. In some cases, the data analysis process can even be regarded as fun. 

As you know, the Head Start Program Performance Standards (HSPPS) are the basic 
framework for program operations. In addition, Head Start program operations are 
guided by other federal and state regulations, local licensing requirements, community 
factors, and the goals/mission of the agency. Therefore, a preliminary step in the data 
analysis process is to select and train a team to carry out the process (refer to Section IV., 
for a discussion of “Who should be at the table”?). 

Because the HSPPS present minimum standards for program services and provide 
guidance for the implementation of the various content areas, a process for data collection 
and analysis should be consistent with these and other regulations. More specifically, 
these standards are the basis for the first step in the data analysis process - forming one or 
more specific questions to be examined. 


Process Component #2. Question(s): 

What Do We Want To Know? 

Before effective data collection or analytical procedures can proceed, one or more specific 
questions should be formulated. These questions serve as the basis for an organized 
approach to making decisions: first, about what data to collect; and second, about which 
types of analysis to use with the data. 

Some questions are close-ended and therefore relatively straightforward, e.g., “Did our 
program meet the 10% mandate for serving children with disabilities last year”? Other 
questions are highly open-ended, such as: “How could we do a better job of parent 
involvement?” In the first case, there are only two possible answers to the question: “Yes” 
or “No.” In the second case, a possible answer to the question could include many relevant 
pieces of information. Many real-life questions that program staff face in the course of 
their work fall somewhere in between these two extremes. 
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Different types of questions require different types of data - which makes a difference 
in collecting data. In any case, the selection of one or more specific questions allows 
the process of data collection and analysis to proceed. Based on the nature and scope 
of the questions (i.e., what is included in the question) programs can then create a 
plan to manage and organize the next step in the process - data collection. Finally, by 
formulating specific questions at the beginning of the process, programs are also in a 
position to develop skills in evaluating their data analysis process in the future. 


Process Component #3. Data Collection: 

What Information Can Help Us Answer Our Question(s)? 

Data collection is a process in and of itself, in addition to being a part of the larger whole. 
Data come in many different types (see the discussion in Section III.), and can be collected 
from a variety of sources, including: 

• Observations • Documents 

• Questionnaires • Tests 

• Interviews • Others 


The value of carefully selecting the questions to be examined is therefore of major 
importance: the way that the question is worded is the foundation for an effective data 
collection plan. We urge programs to develop a specific planning process for data 
collection (no matter how brief) in order to avoid the common pitfalls of the collection 
process, which include having: 

• Too little data to answer the question; 

• More data than is necessary to answer the question; and/or 

• Data that is not relevant to answering the question. 
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In order to successfully manage the data collection process, programs need a plan that 
addresses the following: 

• What types of data are most appropriate to answer the questions? 

• How much data are necessary? 

• Who will do the collection? 

• When and Where will the data be collected? 

• How will the data be compiled and later stored? 

By creating a data collection plan, programs can proceed to the next step of the overall 
process. In addition, once a particular round of data analysis is completed, a program 
can then step back and reflect upon the contents of the data collection plan and identify 
“lessons learned” to inform the next round. 
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Process Component #4. Data Analysis: 

What Are Our Results? 

Once data have been collected, the next step is to look at and to identify what is going on 
- in other words, to analyze the data. Here, we refer to “data analysis” in a more narrow 
sense: as a set of procedures or methods that can be applied to data that has been collected 
in order to obtain one or more sets of results. A list of specific analytical procedures and 
methods is provided below. 

Because there are different types of data, the analysis of data can proceed on different 
levels. The wording of the questions, in combination with the actual data collected, have 
an influence on which procedure(s) can be used - and to what effects. 

The task of matching one or more analytical procedures or methods with the collected 
data often involves considerable thought and reflection. As a GAO report puts it, 
“Balancing the analytic alternatives calls for the exercise of considerable judgment.” This 
is a rather elegant way of saying that there are no simple answers on many occasions. 


Process Component #5. Interpretation: 

What Do The Results Mean? 

Once a set of results has been obtained from the data, we can then turn to the 
interpretation of the results. 

In some cases, the results of the data analysis speak for themselves. For example, if a 
program’s teaching staff all have bachelor’s degrees, the program can report that 100% of 
their teachers are credentialed. In this case, the results and the interpretation of the data 
are (almost) identical. 

However, there are many other cases in which the results of the data analysis and the 
interpretation of those results are not identical. For example, if a program reports that 
30% of its teaching staff has an AA degree, the interpretation of this result is not so clear- 
cut. 

In this case, interpretation of the data involves two parts: 1) presenting the result(s) of the 
analysis; and 2) providing additional information that will allow others to understand the 
meaning of the results. In other words, we are placing the results in a context of relevant 
information. Obviously, interpretation involves both decision making and the use of 
good judgments! We use the term results to refer to any information obtained from using 
analysis procedures. We use the term findings to refer to results which will be agreed 
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upon by the data analysis team as best representing their work. In other words, the team 
may generate a large number of results, but a smaller number of findings will be written 
up, reported, and disseminated. 

On a final note, it is important to state that two observers may legitimately make different 
interpretations of the same set of data and its results. While there is no easy answer to 
this issue, the best approach seems to be to anticipate that disagreements can and do 
occur in the data analysis process. As programs develop their skills in data analysis, they 
are encouraged to create a process that can accomplish dual goals: 1) to obtain a variety 
of perspectives on how to interpret a given set of results; and 2) to develop procedures or 
methods to resolve disputes or disagreements over interpretation. 


Process Component #6. Writing, Reporting & Dissemination: 

What Do We Have To Say? How Do We Tell The Story of Our Data? 

Once data have been analyzed and an interpretation has been developed, programs face 
the next tasks of deciding how to write, report, and/or disseminate the findings. 

While it is not the purpose of this Handbook to provide guidance on writing and 
reporting, we can offer several basic suggestions. First, good writing is structured to 
provide information in a logical sequence. In turn, good writers are strategic - they use a 
variety of strategies to structure their writing. One strategy is to have the purpose for the 
written work to be clearly and explicitly laid out. This helps to frame the presentation and 
development of the structure of the writing. Second, good writing takes its audience into 
account. Therefore, good writers often specify who their audience is in order to shape 
their writing. A final thought is to look upon the writing/reporting tasks as opportunities 
to tell the story of the data you have collected, analyzed, and interpreted. From this 
perspective, the writing is intended to inform others of what you - the data analysis team 
- have just discovered. 
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Process Component #7. Evaluation: 

What Did We Learn About Our Data Analysis Process? 

The final step of the data analysis process is evaluation. Here, we do not refer to 
conducting a program evaluation, but rather, an evaluation of the preceding steps of the 
data analysis process. Here, program staff can review and reflect upon: 

• Purpose: was the data analysis process consistent with federal standards and other, 
relevant regulations? 

• Questions: were the questions worded in a way that was consistent with federal 
standards, other regulations, and organizational purposes? Were the questions 
effective in guiding the collection and analysis of data? 

• Data Collection: How well did the data collection plan work? Was there enough time 
allotted to obtain the necessary information? Were data sources used that were not 
effective? Do additional data sources exist that were not utilized? Did the team collect 
too little data or too much? 

• Data Analysis Procedures or Methods: Which procedures or methods were chosen? 
Did these conform to the purposes and questions? Were there additional procedures 
or methods that could be used in the future? 

• Interpretation/Identification of Findings: How well did the interpretation process 
work? What information was used to provide a context for the interpretation of the 
results? Was additional relevant information not utilized for interpretation? Did team 
members disagree over the interpretation of the data or was there consensus? 

• Writing, Reporting, and Dissemination. How well did the writing tell the story of the 
data? Did the intended audience find the presentation of information effective? 

See Appendix C for a checklist on how to evaluate your data analysis process. Section 
V walks you through several content examples using all of the process components 
identified above. 

In sum, data analysis is a process: a series of connected activities designed to obtain 
meaningful information from data that have been collected. As the graphics demonstrate, 
the process can be conceptualized in different ways (linear or cyclical). However, we 
strongly believe that either approach can be effective if each of the individual components 
of the process are included. In turn, each part of the process is based on decision making. 
Each stage of the process includes decision making; which decisions are made will then 
influence the remaining stages of the process. 


© AED/TAC-12 Spring 2006. Permission granted to copy for non-commercial uses only. 


IV. Managing the Data Analysis 
Process 


Developing a Team 

Similar to the Community Assessment, in which the Migrant and Seasonal Head Start 
Director has the responsibility of ensuring that the data analysis is fully completed and 
implemented, we would suggest the same for the data analysis. Ultimately the Migrant 
and Seasonal Head Start Director is responsible for assuring that the data be gathered 
and analyzed, is ready to present to appropriate stakeholders, and ready to be used for 
program decision making and enhancement. 

Although the director holds the final responsibility for the data analysis process and 
products, other staff are active participants in the collection and analysis of data. 
Programs may choose to use existing policy groups to implement the data analysis 
process, or establish a separate data analysis team. The Policy Council and Board are 
consulted regarding the plan for implementing the decisions ascertained from the data 
analysis, and ultimately both groups review and approve all decisions that are made. 

The data analysis process must be designed to include a variety of stakeholders. 
Stakeholders are individuals that have a vested interest in the Migrant and Seasonal 
Head Start Program - they will benefit from the program in some way. Migrant and 
Seasonal Head Start programs generally operate using a systems approach, which means 
that there already exist several teams and committees made up of various levels of the 
organization including staff, parents, Board and Policy Council and representatives from 
the community. Various individuals can be identified to form a data analysis team and 
select a team leader. 

The data collection process can be conducted using a variety of teaming models, we 
recommend the use of two models. The first model emphasizes the use of members 
from existing committees (i.e., HSAC, Ed. Comm., CA Comm.) and the second model 
emphasizes the creation of a special data analysis team. Both of these models use a team 
approach. The team approach is more effective because it takes the burden off of a single 
individual and distributes the effort among others, thereby strengthening the process. It 
also ensures a more comprehensive approach to the data analysis process. It is essential to 
match responsibilities with individual interests and abilities. 

In both models, staff, usually the director and service area managers, completes the tasks 
of collecting, analyzing, presenting the data as well as evaluating the data analysis process. 
Both models require the scheduling of regular meetings to discuss the work and progress 
of the data analysis process. It is helpful to include team members who have experience in 
data analysis so that they can mentor others on the team. 

Staff from all program areas have important information and insights to contribute. The 
information gathered should reflect a perspective that encompasses all aspects of the 
Migrant and Seasonal Head Start program. 
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All team members participating in the data analysis process will need to be trained. For 
optimal results, try to match tasks with individual abilities and interests. 

Migrant and Seasonal Head Start parents are an important source of information and 
key advisors in the process. Their participation includes acting as: 1) decision makers as 
members of policy groups or committees, 2) channels of information and opinions from 
eligible families, 3) collectors of information and opinions from their peers. Parents have 
information regarding trends in local agriculture, local migration patterns, child health 
status and service needs that impact Migrant and Seasonal Head Start planning and 
programming. 

Community partners and other agencies often have their own data analysis needs. 
Information from community partners can be extremely valuable. Some community 
partners might have systems already in place to collect and analyze data (e.g. community 
colleges, migrant/ community health clinics, agricultural extension offices, health 
departments, other Head Start programs, etc.). 

The Board, Policy Council and advisory committees should be kept informed regarding 
the progress. Whenever possible, discuss the findings with appropriate decision making 
bodies before any document is completed to be submitted or for dissemination. This will 
enrich the process and enhance participation from appropriate individuals (i.e. parents) 
in the decision making process. 

If a consultant(s) are on the team, they can work with the leader. If the organization does 
not have the expertise to conduct the process, the program may want to request assistance 
including training of the team from their Head Start technical assistance provider, another 
agency familiar with data analysis and/or a private consultant who knows Head Start and 
has worked with similar agencies. Summer interns can also benefit from participating in 
this process and can be of great help. 


Important Tip: Sometimes when working with a person outside the MSHS or HS com- 
munity there can be a disconnect between their training/background and the purpose of 
your task. Always make sure that the individual or agency hired or asked to do the task, 
understand what you want and what you will be doing with the information, as well as the 
population that you serve. 
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Planning 

Planning is a very integral part of the data analysis process. It is imperative that you 
develop a plan to engage in data analysis. The plan should outline your data analysis 
process and include the purpose, questions, data collection methods, needed resources, 
identified lead person, as well as a timetable for completing all of the necessary tasks. The 
reality within most programs is that individuals are always managing a variety of different 
tasks at any given time. The actual process of data analysis should be given priority. 

Allow time for the identified person to have time or the person’s supervisor can reassign 
certain tasks so that they can devote the appropriate amount of time to the project. 


Sample Planning Table 


Purpose 

Questions 

Data 

Collection 

Methods 

Needed 

Resources 

Lead Person 

Timeframe 

Purpose 1 






Purpose 2 






Purpose 3 







The planning piece of the framework gives you an opportunity to identify the resources 
that you will need. Some examples of needed resources include the translation of key 
pieces of information for parents, additional clerical support and scheduling meetings in 
conjunction with other activities. 

As with other planning pieces within Head Start it is important that you consult with 
the Policy Council (PC) regarding the plan prior to its implementation. Solicit ideas and 
assistance from PC members regarding the proposed data analysis process. 


Important Tip: Traditionally many programs find themselves in a crunch when dealing with 
data analysis. Taking time to plan out your process will save you time in the long run. 
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Key Points to remember about planning: 

• As part of your plan include a strategy for keeping the Board of Directors, Policy 
Council and other key players updated regarding the progress of the data analysis 
process. 

• Identify in advance the resources that you will need to implement the process. Some 
examples include the translation of key pieces of information for parents, additional 
clerical support and scheduling meetings in conjunction with other activities. 

• Consult with the Policy Council (PC) regarding information being derived from the 
data analysis process as well as decisions that are being made as a result of the process. 
Solicit ideas and assistance from PC members regarding the proposed data analysis 
process. 

• Ensure that all data analysis team members are oriented and trained to understand the 
importance of the data analysis and its impact on the provision of Head Start services 
to migrant and seasonal children and families. 

Data Management 

Data comes from many places and a variety of formats. It is important to take a 
systematic approach and think about the following questions: 

• How will you keep track of all the data? 

• How will you make the data available to several members of your team? 

• How will you collect and integrate data over a period of time? 

• How can you set up the system so that others can step in if your job responsibilities 
change? 

Choices about which data to include in your Data Analysis is highly dependent on both 
the availability of data and the quality of the available data. For example if we look at 
the PIR, it would be very informative to report on data on otitis media (ear infection) of 
children enrolled in your program. Since data on otitis media is not readily available in 
the PIR, however, information on hearing condition is reported, which can also provide 
interesting information. Design your data collection so that you gather a balance of 
internal, community, state and national information that relates to your program’s 
purpose and to the population that constitutes your eligible families and the community 
that they become part of when they migrate into your service area. 

With regards to data management we strongly encourage that you store the information 
on the computer to the best of your potential and that you back up your data regularly. 
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Writing 

Write your key findings and interpretations as related to your purpose. A well written 
report ought to proceed from your findings and interpretations to recommendations 
that are clear and concise. The report should include a brief narrative consisting 
of a restatement of your purpose(s), question(s), finding(s), interpretation (s) and 
recommendation(s). The following is list of useful tips to keep in mind when writing a 
report: 

• Write to your audience; 

• Reference all tables and data; 

• Title all figures, maps and tables; 

• Connect figures, maps and tables to narrative; 

• When using figures, maps and tables make sure that they do not contradict the 
narrative; 

• Make sure that your recommendations are derived from your data and not “random” 
stand alone pieces that have not been introduced earlier; and 

• It is highly recommended that your narrative include a brief description of your 
methodology. 

Evaluation 

As identified in Section 3 Conceptualizing Data Analysis as a Process, the final step of 
the Managing the Data Analysis Process is evaluation. Here, the data analysis team can 
review and reflect upon: 

• Development of a team: was team structure and team make-up appropriate? 

• Planning: was the plan followed, timelines met? 

• Data Management: How well was the data stored? Did the appropriate people have 
access to data when needed? Was the appropriate computer software used, was it user 
friendly and were you able to go about finding needed information? 

• Writing, Reporting: How well did the writing tell the story of the data? Did the 
intended audience find the presentation of information effective? Was the report well 
written? 


26 


© AED/TAC-12 Spring 2006. Permission granted to copy for non-commercial uses only. 


V. Data Analysis Using 
Content Area Examples 


Introduction to Data Analysis 

Working with data in real life situations always involves some combination of good news 
and bad news (of course, not necessarily in equal amounts). 

In Head Start programs, data is everywhere. Every enrollment form, developmental 
history and child screening is a potential source of data, as are lesson plans, teachers’ 
observations of children and even the comments made by bus drivers. Likewise, every 
pre-service evaluation form is also a source of data, as is every statement made by a 
parent, whether or not it is written down. 

As we have mentioned previously, in Head Start programs, “data” come from many 
sources and are present in large amounts. Therefore, the good news is that we have plenty 
to work with! Second, all staff in MSHS already possess many skills in analyzing data. 
Consider the following example: 

Q: Which size raise would you prefer? 

a) 0.5% 

b) 1.5% 

c) 5% 

d) 15% 

e) 50%? 

Congratulations! Everyone got the correct answer! (Remember, the question asked what 
raise you would prefer . . . .not what you should expect) . 

Some of the “bad” news connected with data analysis includes: 1) getting used to the idea 
that there are different types of data; 2) keeping track of the many “ways” of working 
with data; and 3) becoming comfortable with the reality that there is lots to learn if you 
are new to this. Actually, the related fields of data analysis are so broad and continuously 
developing that even people with decades of experience can always learn more - if they 
want to. (PS- there’s a lot more bad news but we have space limitations ) 

Developing data analysis skills means using the skills that you already have and practicing 
the use of new skills . . . until they become familiar. Accordingly, we use this section to 
guide participants through some of the basic procedures and methods available for 
the analysis of data. Since there is a basic distinction between quantitative data (think, 
numbers) and qualitative data (think, no numbers) there is also a difference in what “data 
analysis” means in practice, depending upon the type(s) of data you are (or soon will be) 
working with. 
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Therefore, this section is divided into the following sub-sections: 1) Terms & Concepts; 2) 
Procedures and Methods; and 3) Content Area Examples. 

Terms & Concepts: There is a universe of terms and concepts related to data and its 
analysis. The terms and concepts are the “raw materials” or basic ingredients of the data 
analysis process. There are on-line glossaries that exceed 30 pages just for quantitative 
items; in this sub-section we have developed our list to include terms and concepts for 
both quantitative and qualitative purposes. In addition, we have limited our list to those 
terms and concepts that we believe are crucial to developing an initial understanding of 
basic procedures. 

Procedures and Methods: There are a large number of procedures and methods available 
for data analysis. A key consideration is that none of these procedures are necessarily 
abstract. That is, the same procedures can be applied across a wide range of settings - 
from baseball games to shopping malls to educational settings. Knowledge of data analysis 
procedures or methods means that we have the means to work with data; procedures and 
methods allow us to “make sense” of the data for program purposes. Since there are two 
basic types of data - quantitative and qualitative - we use this distinction to organize our 
presentation of procedures and methods in this sub- section. 

Content Area Examples: Content area examples are where general, abstract procedures 
or methods meet up with specific data from Head Start program content areas. In this 
sub-section we provide several examples of more “simple” analyses followed by some 
more in-depth examples. Again we use the basic distinction between quantitative and 
qualitative data to sort out the content area examples. 
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Terms and Concepts 


Term 

Definition 

Analysis 

noun: an investigation of the component parts of a whole and their 
relations in making up the whole 

noun : the abstract separation of a whole into its constituent parts in 
order to study the parts and their relations 

Code 

noun: a category deemed important by the an individual(s) 
conducting the analysis. It is a method used to label important 
pieces of information that are contained in the narrative. 

Correlation 

noun: a statistical relation between two or more variables such that 
systematic changes in the value of one variable are accompanied by 
systematic changes in the other 

noun: a statistic representing how closely two variable co-vary; 
it can vary from -1 (perfect negative correlation) though 0 (no 
correlation) to +1 (perfect positive correlation) (Example: “What is 
the correlation between those two variables?”) 

Data 

noun: a collection of facts from which conclusions may be drawn 
(Example: “Statistical data”) 

Denominator 

noun: the divisor of a fraction 

Difference 

noun: the number that remains after subtraction; the number that 
when added to the subtrahend gives the minuend 

noun: a variation that deviates from the standard or norm 

Estimation 

noun: a judgment of the qualities of something or somebody 
(Example: “In my estimation the boy is innocent”) 

noun: an approximate calculation of quantity or degree or worth 
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Term 


Definition 


Interpretation 

noun: an explanation of something that is not immediately obvious 
(Example: “The edict was subject to many interpretations”) 

noun: an explanation that results from interpreting something 
(Example: “The report included his interpretation of the forensic 
evidence”) 

Interview 

noun: the questioning of a person (or a conversation in which 
information is elicited); often conducted by journalists (Example: 
“My interviews with teenagers revealed a weakening of religious 
bonds”) 

verb: discuss formally with (somebody) for the purpose of an 
evaluation (Example: “We interviewed the job candidates”) 

Narrative 

noun: a message that tells the particulars of an act or occurrence 
or course of events; presented in writing or drama or cinema or 
as a radio or television program (Example: “His narrative was 
interesting”) 

adjective: consisting of or characterized by the telling of a story 
(Example: “Narrative poetry”) 

Numerator 

noun: the dividend of a fraction 

Mean 

noun: an average of n numbers computed by adding some function 
of the numbers and dividing by some function of n 

Median 

noun: the value below which 50% of the cases fall 

adjective: relating to or situated in or extending toward the middle 

Mode 

noun: the most frequent value of a random variable 

Percentage 

noun: a proportion multiplied by 100 
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Term 


Definition 


Qualitative 

adjective : involving distinctions based on qualities (Example: 
“Qualitative change”) 

adjective : relating to or involving comparisons based on qualities 

Quantitative 

adjective: expressible as a quantity of relating to or susceptible 
of measurement (Example: “Export wheat without quantitative 
limitations”) 

adjective: relating to the measurement of quantity (Example: 
“Quantitative studies”) 

Questionnaire 

noun: a form containing a set of questions; submitted to people to 
gain statistical information 

Reliability 

noun: the trait of being dependable or reliable 

Standard 

Deviation 

noun: the square root of the variance 

Statistics 

noun: a branch of applied mathematics concerned with the 
collection and interpretation of quantitative data and the use of 
probability theory to estimate population parameters 

Statistics 

(Descriptive) 

noun: a branch of statistics that denotes any of the many techniques 
used to summarize a set of data. In a sense, we are using the data 
on members of set to describe the set. 

Statistics 

(Inferential) 

noun: comprises the use of statistics to make inferences concerning 
some unknown aspect (usually a parameter) of a population. 

Sum 

noun: the whole amount 

noun: a quantity obtained by addition 
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Term 


Definition 


Survey 


Themes 


Trend 

Validity 


Variance 


noun : short descriptive summary (of events) 

verb: look over in a comprehensively, inspect (Example: “He surveyed 
his new classmates”) 

verb: make a survey of; for statistical purposes 

noun: a unifying idea that is a recurrent element within an interview 
or a narrative leading to a set of patterns. There is no agreed-upon 
methodology in narrative analysis to derive themes from patterns. 
One practice, however, is to use an analysis team, with “themes” being 
whatever sets of “like” information the team reaches consensus on, 
based on discussion of transcripts and analysis of patterns. 

noun: the subject matter of a conversation or discussion 

verb: a general direction in which something tends to move (Example: 

“The trend of the stock market”) 

noun: the quality of having legal force or effectiveness 

noun: the quality of being logically valid 

noun: the second moment around the mean; the expected value of the 
square of the deviations of a random variable from its mean value 

noun: the quality of being subject to variation 
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Data Procedures and Methods 


Procedures for Data Analysis - Quantitative 

There are a variety of procedures that may be used to analyze quantitative data. The two 
most basic types are: 

• Summary Measures 

• Variance Measures 

Summary Measures 

Questions to consider when working with summary measures include: 

• How do the data converge (come together)? 

• What is a “typical” (average) value? 

• Where is the middle (center) of a group? 

Types of Summary Measures: 

1 . Mean 

• Mean = “Arithmetic average” of the scores 

• Example: scores = 64, 70, 80, 80, 90, 98, 100 

Mean = Sum of values divided by number of scores: 

64 + 70 + 80 + 80 + 90 + 98 + 100 = 582 
582 ^7 = 83.14 
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2. Median 

Median = score in the middle; 50th percentile 
Example #1: 

• Median scores = 64, 70, 80, 80, 90, 98, 100 

If # scores = odd: Find the score in the middle 
64, 70, 80, [80], 90, 98, 100 

Median = 80 

Example #2: 

• Median scores = 64, 70, 80, 80, 90, 98, 100, 100 
If # scores = even: Average the 2 in the middle 

64, 70,80, [80,90,] 98, 100, 100 
80 + 90= 170 
170-2 = 85 
Median = 85 

3. Mode 

• Mode = largest number of scores 
Example #1: 

• Example: scores = 64, 70, 80, 80, 90, 98, 100 

Mode = 80 

Example #2: 

• Example: scores = 64, 70, 80, 80, 90, 98, 100, 100 

Modes = 80, 100 
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Important Tip: Some of these procedures can be performed manually. Many people 
confuse data analysis with complicated computer programs but simple data analysis 
procedures involve using a pencil, note pad and a calculator. Other procedures can 
be accomplished using features of Excel software. (See Appendix B) 


Variance Measures 

Questions to consider when working with variance measures include: 

• How do scores differ? 

• What are the differences between individuals in a group? 

• What is the range of outcomes? 


Example of variance measures: 

50 participants rate a training using 

I acquire new knowledge and/or skills 

Strongly Disagree Disagree Agree Strongly Agree 

□ □ □ □ 


A. Results to the statement: I acquired new knowledge and/or skills. 

Strongly Disagree Disagree Agree Strongly Agree 

45 3 1 1 

In this example the variance is small (or, low variability). Staff strongly disagreed and felt 
that the training did not lead them to acquire new knowledge and/or skills. 

B. In a second training consider the responses to the statement: I acquired new knowledge 
and/or skills. 

Strongly Disagree Disagree Agree Strongly Agree 

0 5 5 40 

As in the previous example the variance is small (or, low variability). But unlike the 
previous example, staff strongly agreed and felt that the training lead them to acquire new 
knowledge and/or skills. 
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C. In a third training consider the responses to the statement: I acquired new knowledge 
and/or skills. 

Strongly Disagree Disagree Agree Strongly Agree 

15 13 14 8 

Unlike the previous two examples the variance in this example is large (or, high 
variability) . In this example staff were divided opinions as to whether they acquired new 
knowledge and/or skills. 

There are many ways to convey the same information. Below are three additional ways to 
display the same data. 


I acquired new knowledge and/or skill: 


Strongly Disagree 

15 

Disagree 

13 

Agree 

14 

Strongly Agree 

8 


50 


I acquired new knowledge and/or skill: 


Strongly Disagree 

30% 

Disagree 

26% 

Agree 

28% 

Strongly Agree 

16% 


100% 


(n=50) 

1 acquired new knowledge and/or skill: 


16% 

/'-'"pv 30% 

28 %V / y 


26% 


□ Strongly Disagree 
■ Disagree 

□ Agree 

□ Strongly Agree 
(n=50) 
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The importance of measuring variance is that it demonstrates differences. Once the 
difference is identified, then you can seek an explanation and determine if it is significant. 
In the above examples, you might determine that the first training was not valuable to 
staff and you may reconsider the topic or the person delivering the training; the second 
training was valuable and you could consider a follow-up and continue to provide the 
training; for the third training you could consider the target audience and re-evaluate the 
material being covered and appropriateness for some staff such as teachers versus family 
service workers. In these cases further information may be needed as to why there was low 
or high variability in the responses. 

Methods for Data Analysis - Qualitative 

There are also a number of ways that qualitative data can be analyzed. We will concentrate 
on a narrative analysis method using questionnaires and interviews. 

Although the following examples are identified as narrative analysis the same process can 
be used when analyzing interviews and questionnaires. 

In the example below we will explore how the transcript can be used as a source of data. 

Sample Qualitative Data: Transcript from a 
2002 Parent Interview Conducted in a Rural 
Head Start Program in Upstate New York 

(Family one - husband) 

OK, well, me first, before anything, I came here because of the poverty, do you 
understand me? And for a good quality of life, in my country not so much 
thinking about myself but about my smaller sisters so that they can have a 
better education considering that I did not have any, and another reason was 
in my country there are no good jobs, and very little work. They pay you very 
little and you never leave the poverty and another reason is that my father 
had two brothers that were already here and I thought that the more of us that 
are here the better that everybody could help each other out and it would be 
easier to get ahead with our younger sisters. 

(Family one - wife) 

The same to help my parents I came and I would send money and then my 
brother came and then the same. 

(Family one - husband) 

Because in her case, she or better yet sometimes us (in our situation) because 
we were the eldest, we were men but in her case the eldest are women, and the 
ones that were going to work, theoretically to help the parents the most were 
the youngest. 
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(Family one-wife) 

My dad was already here when I came, he was here, mi mother was in Mexico, 
my dad stayed here for some time and than he went to Mexico. 

(Family one-husband) 

More than anything, my parents for example they are that type of people 
that support you no they never stop, for example your dreams do not become 
reality they always try that if you decide, you know who you are, and they 
always let us what we wanted better yet things that they showed us and that 
were good we would do them and they always tried to better us. 

The first step in analyzing qualitative data is to arrange the data in a manageable format 
that will facilitate the process of assigning codes and themes to sections of the transcript: 
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Text 

Code 

Theme 

OK, well, me first, before anything, 
I came here because of the poverty, 
do you understand me? 

Poverty 

Purpose for migration 

And for a good quality of life, in 
my country not so much thinking 
about myself but about my smaller 
sisters so that they can have a better 
education considering that I did not 
have any 

Family Life 

Better 

education 

Purpose for migration/ 
family 

and another reason was in my 
country there are no good jobs, and 
very little work. 

Work 

Purpose for migration 


Once the theme has been established, count the number of times that the theme was 
identified. At this point the themes can be arranged in order of frequency (the number of 
times that the theme was identified) . 


Theme 

Frequency 

Strength 

Barrier 

Purpose for migration 

2 



Purpose for migration/ 
family 

1 




After using the above format for managing narrative data you may want to determine 
if the theme is a strength or a barrier. The identification of the theme as a strength or a 
barrier will help in prioritizing as well as for planning purposes. 


Note: To review the definition of code and theme, see page 29 and 32. 
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Content Area Analysis Using Different Procedures and 
Methods 


On the following pages we walk you through the data analysis process using five of the key 
process components discussed earlier in Section III. Content examples include: 

• Health 

• Teacher Credentials 

• Classroom Quality 

• Disabilities 

• Estimating the Number of Eligible Migrant and Seasonal 
Head Start Children and Families 

• Infant Toddler Language 

• Family Strengths and Needs 
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Health (PIR Medical Services) 


Process Components 

Purpose 

Question 
Data Collection 


Content Information 

To identify the top medical conditions that impact children 
in MSHS programs over a three year time period, using the 
Program Information Report (PIR). 

What is the number one medical condition that impacts 
MSHS children enrolled in MSHS programs over a three year 
period? 

Using the PIR data - Percentage of children who received 
treatment for main health conditions: 


MSPB-03 

MSPB-04 

MSPB-05 

Anemia 

3.1% 

2.5% 

2.8% 

Asthma 

2.0% 

2.0% 

2.3% 

Hearing Diff 

1.7% 

1.7% 

1.7% 

Overweight 

2.8% 

3.2% 

3.6% 

Vision Problem 

1.3% 

1.3% 

1.4% 
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Health (PIR Medical Services) (continued) 

Process Components Content Information 


Data Analysis 


Interpretation 


The above percentages are proportions that will tell us 
what fraction of the total number of children received 
treatment for the one the identified health conditions. 

The percentages were calculated by taking the number 
of children with one of the main health conditions and 
dividing that number by the actual enrollment. 

Add the percent of children identified with each particular 
health condition for the years available to you (in this 
example the years are 2003, 2004, 2005), and divide the total 
by the number of years being analyzed: 


Anemia 

Total 

8.4% 

# of years 
3 

Average 

2.8% 

Asthma 

6.3% 

3 

2.1% 

Hearing Diff 

5.1% 

3 

1.7% 

Overweight 

9.6% 

3 

3.2% 

Vision Problem 

4% 

3 

1.3% 

At this time it is recommended that a visualizing strategy be 
used. Rank the health condition using the average. 


Average 

Rank 


Overweight 

3.2% 

1 


Anemia 

2.8% 

2 


Asthma 

2.1% 

3 


Hearing Diff 

1.7% 

4 


Vision Problem 

1.3% 

5 



Being overweight is the number one medical condition that 
impacts MSHS children enrolled from 2003-2005. 
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Teacher Credentials 


Process Components 

Content Information 

Purpose 

Since the 1998 Head Start Act reauthorization, Head Start 
has been mandated to have least 50% of its teachers hold 
an AA degree or higher. In addition, programs are also 
mandated to have at least one qualified teacher in each 
classroom (i.e., the teacher possesses a CDA or higher). In 
order to monitor and evaluate how their program is meeting 
these mandates, MSHS staff can make use of several data 
analysis procedures. 

Question 

Program staff could begin by asking the question: What 
is the current level of teacher credentials in our program? 
Another way to frame the question would be to ask: What 
percentage of classroom teachers employed by our program 
now meet the credentialing mandate? 

Data Collection 

Since a “credentialed” teacher is someone with an AA degree 
or higher, two pieces of data (information) are needed: 1) 
the total number of teachers with either an AA degree, BA 
degree or higher; and 2) the total # teachers. 

Data Analysis 

The analysis of this data is simply to divide: 
Total # teachers with AA, BA, or higher 
Total # teachers 


X % credentialed teachers. 
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Teacher Credentials (continued) 

Process Components Content Information 


Interpretation 


Consider the following two examples: 

Program A 

Teachers with AA degree = 37 
Teachers with BA degree =11 
Teachers with MA degree = 1 

Total teachers with degrees = 49 

Total number of teachers = 50 
Credentialed teachers = 98% 

Program B 

Teachers with AA degree = 37 
Teachers with BA degree =11 
Teachers with MA degree = 1 

Total teachers with degrees = 49 

Total number of teachers = 225 
Credentialed teachers = 22% 


Teacher Credentials (continued) 

Process Components Content Information 


Interpretation 

continued 


Teachers with a CDA = 50 
Teachers with AA degree = 37 
Teachers with BA degree =11 
Teachers with MA degree = 1 
Total number of qualified teachers = 99 
Total number of teachers =100 
Qualified teachers = 99% 


The same process can be used to identify the percent of 
teachers who are qualified. Since a “qualified” teacher is 
someone with a CDA or higher, the calculations would look 
like this: 


Once the calculations have been done, program staff can 
consider the following: 

1. Does the result “speak for itself ”?- or, 

2. Does the result need to be presented in the context 
of additional information in order to be “adequately” 
understood? 

In the case of teacher credentials for Program A, the result 
pretty well speaks for itself. The program far exceeds the 
Head Start Bureau mandate for teacher credentials. 
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Teacher Credentials (continued) 

Process Components Content Information 


Interpretation 

continued 


In the case of Program B, if 22% of a program’s teaching 
staff are credentialed, the interpretation of this result may 
vary, depending upon other, relevant information. For 
example, say the program supported a large number of 
teaching staff (n = 20) to obtain degrees. However, if 18 out 
of 20 teachers who received their degrees left to take jobs in 
public school programs, this would decrease the percent of 
credentialed teachers. Other relevant factors could include 
the presence or absence of higher education partners, 
changes in access to distance learning, and/or the level of 
staff turn- over. Program staff can consider and discuss all 
relevant factors are part of their interpretation of the data. 

Alternatively, an analysis of trends in the data might provide 
meaningful interpretation (explanation) of the current level 
of teacher credentials. 


Program B 

2002 

2003 

2004 

2005 

% teachers with credentials 

5% 

11% 

15% 

22% 


In this example, the program is still below the 50% mandate, 
but presents evidence of 

(substantial) progress between 2002 - 2005. 

The teacher credential mandate presents one area in which 
programs can use and develop their skills in data analysis 
and interpretation. The goal is to use these skills to obtain 
a clear and valid picture of where the program is “at” - and, 
possibly, where it has been. 
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Classroom Quality 


Process Components Content Information 


Purpose 


To develop an initial “portrait” of classroom quality. 


Question 


What is the current level of quality in our classrooms? 


Data 


Possible data sources include: 


Early Childhood Environments Rating Scale, Revised 
Edition (ECERS-R) (Harms, Clifford, & Cryer, 1998). 
http:// www. fpg . unc.edu/ ~ ecers/ 

Early Language & Literacy Classroom Observation Tool 
(ELLCO) (Smith, Dickinson, Sangeorge & Anastasopoulos, 
2002 ). 

http :// www.brookesp ublishing . com 

Regardless of which specific instrument is used to measure 
classroom quality, several common features are: 

• Personnel must be trained to use the instrument; 

• Different aspects of classroom quality are measured by 
assessing specific items, using a scale that is specified in 
the instrument manual; and 

• Scores for various sub- scales and a total score can be 
calculated. 

This last feature provides several options for data analysis. 
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Classroom Quality (continued) 

Process Components Content Information 


Data Analysis 


The different instruments for measuring classroom quality 
use information from specific items to form “portraits” of 
different aspects of overall classroom quality (sub-scores 
or sub-scales). In addition, the instrumental sub-scales 
combine to form a total (overall) score. Therefore, scores 
obtained by using one (or both) of the instruments can be 
examined at two basic levels: 


Instrument-level Options: 

1 . Individual item 

2. Sub-group (sub-scale) 

3. Total score 


Program-level Options: 

1. Individual classroom 

2. Classrooms across center 

3. Centers across agency 

4. Grantee-wide 


For example, ECERS-R examines the following seven sub- 
scales (categories or groups) of classroom quality: Space and 
Furnishings; Personal Care Routines; Language-Reasoning; 
Activities; Interaction; Program Structure; and Parents and 
Staff. 


Each of these sub-groups is assessed using a 7-point scale, 
providing scores, as follows: 

1 = Inadequate; 2; 3 = Minimal; 4.; 5 = Good; 6.; 7 = 
Excellent. 


Note that the odd numbers of the scale are where the level of 
quality is defined; the even numbers represent “mid-points” 
between two defined levels. 


Once we have a grasp of how the instrument is organized 
and how items are measured, we can begin to understand 
what data we will be gathering and then how to analyze and 
interpret it. 
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Classroom Quality (continued) 


Process Components 

Data Analysis 
continued 


Content Information 

When we take all of the above information into account, we 
can then choose to analyze data along a continuum - 
ranging from a very “micro” level to an overall, “big picture” 
(macro) level: 

Micro Macro 

< > 

At the smallest level of analysis, we could look at a single 
item in one classroom. At the largest level of analysis, we 
could look at the total (instrument) scores for all classrooms. 

In between, we have a range of choices for analysis, 
depending upon our purposes. The chart below presents 
data from an actual preschool classroom in New Jersey that 
participated in a research study in November, 2001 (note: 

Appendix contains a copy of the completed score 

sheet). 


ECERS DATA; NJ Classroom 3-B 

7 -r 



1 2 3 4 5 6 7 


ECERS Sub-Scales 
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Classroom Quality (continued) 

Process Components Content Information 


Data Analysis 
continued 

Remember: 

The 7 sub-scales = 

Sub-scale 1 = Space and Furnishings; Sub-scale 2 = Personal 
Care Routines; Sub-scale 3 = Language-Reasoning; Sub- 
scale 4 = Activities; Sub-scale 5 = Interaction; Sub-scale 6 = 
Program Structure; and Sub-scale 7 = Parents and Staff 

Each sub-scale is scored as: 

1 = Inadequate; 2; 3 = Minimal; 4.; 5 = Good; 

6.; 7 = Excellent. 

Interpretation 

Is there evidence of classroom strengths? Consider the 
following two sub- scales: 

1. Space (score = 5.87 out of a possible 7.0). 

2. Personal Care Routines (score = 5.33 out of a possible 
7.0). 

These sub-groups are identified as strengths because they fall 
within (above) the “good” range 

What other evidence is there? Consider: 

1. All other sub- scales are scored below the “good” level. 

2. The Language-Reasoning score is lowest, (3.0 out of a 
possible7.0) -- falling into the minimal range. 

Note: The Language- Reasoning sub-scale contains four 
specific items that are scored individually and then totaled: 
Books and pictures; Encouraging children to communicate; 
Using language to develop reasoning skills; and Informal use 
of language. Criteria for scoring each item are presented in 
the Instructions section of the ECERS-R. 
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Classroom Quality (continued) 


Process Components Content Information 

This combination of evidence from one classroom can 
lead us to consider the larger picture. For example, since 
the Language- Reasoning sub- scale scores were so low in 
Classroom 3-B, what about the scores from other classrooms 
in the same center or program? A next step is to conduct 
the analysis across classrooms (i.e., how do the different 
classrooms in the center (program) compare to each other?). 


LANGUAGE (across all classrooms) 



Classroom 


What can we interpret from the data in the chart above? 
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Classroom Quality (continued) 

Process Components Content Information 


Interpretation 

continued 


By ranking classrooms on a simple distinction - which 
classrooms are at/above the “good” score (i.e., 5.0) and 
which are below? - we can quickly identify: 

• 3 out of 10 classrooms (30%) are at or above a “good” 
score in the Language-Reasoning sub-scale. 

• 7 out of 10 classrooms (70%) are below a “good” score in 
the Language-Reasoning sub-scale. 

We can also calculate the mean score for all classrooms in 
the Language-Reasoning sub-scale by adding all of the scores 
for individual classrooms (the total = 39) and dividing by 
the number of classrooms (10) for a mean score of 3.9. 

We can also account for the variance in scores by simply 
presenting the range of scores, from highest (e.g., classroom 

• 5 is a perfect 7) to lowest (e.g., both classrooms # 4 and #8 
have a score of 2). A summary statement of these findings 
could be written as: 

The mean for the 10 classrooms on the Language- Reasoning 
sub-scale was 3.9, with a range of 2 - 7. 

We could present an interpretation of these findings along 
these lines: 

The 10 classrooms different substantially in quality in the 
Language- Reasoning sub- scale. Although a few classrooms 
present good or excellent environments for children, the 
majority of classrooms fall below a “good” rating. 
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Classroom Quality (continued) 

Process Components Content Information 


Interpretation 

continued 


1 . The worst is over. 

2. Planning is straightforward. 

3. The lower we start, the only place to go is up. 

ECERS Example: 10 Classrooms 

One advantage of looking at classroom quality data 
by individual classroom is that we can focus upon the 
individual strengths and need for improvement for that 
particular classroom. However, in large-size programs with 
many classrooms, there are several potential disadvantages to 
looking at individual classroom data. First, the process can 
become very time consuming, as classroom data is reviewed 
and analyzed separately. Second, someone reviewing the 
data can get “lost” in the details, and not able to focus on the 
“big picture.” 

One way around these disadvantages is to aggregate 
(combine, put together) data from different classrooms. 

The graphic below presents hypothetical data for all ECERS 
sub-scales for ten classrooms. By aggregating the data, 
everything now fits into one place and is available for 
analysis. 


Obviously, this initial look at classroom quality, focused 
on the Language- Reasoning sub-scale, is not particularly 
uplifting. Unfortunately, when we look at data, we often 
“discover” bad news. However, while the data considered 
above are nothing to cheer over, there are some positive 
aspects. 
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Classroom Quality (continued) 

Process Components Content Information 


Interpretation 

continued 


When data are aggregated, this presents a variety of options 
for analysis. For example, someone reviewing the data 
could look specifically for strengths. Another option would 
be to analyze for areas needing improvement. Yet another 
approach would be to look for patterns in the data - i.e., 
where are there similarities and differences? 

Take a moment to review the chart on the following pages 
and, given the data, see how you would answer the following 
questions: 

• What would you identify as strengths? 

• What are areas needing improvement? 

• What patterns do you see in the data? 

• What else “stands out for you? 
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Disabilities 


Process Components 

Content Information 

Purpose 

To estimate the number of children with a disability prior to 
enrollment in the MSHS program for the upcoming year. 

Question 

What is a meaningful estimate of children identified to have a 
disability prior to enrollment in MSHS programs using prior 
PIR data? 

Data Collection 

The data that is need is the following: 

# of Children with disabilities prior to enrollment for 

2001,2002,2003,2004 

Actual Enrollment for 2001, 2002, 2003, 2004 

Total Number of Children Identified with a Disability Prior 
to Enrollment for HS Region “XX” from 2001-2004 

2001 = 846 

2002 = 927 

2003 = 1,059 
2004= 1,099 

Actual Enrollment for HS Region “XX” from 2001-2004 

2001 =28,157 

2002 = 30,498 

2003 = 33,188 

2004 = 33,023 
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Disabilities (continued) 

Process Components Content Information 


Data Analysis 


Take the # of children with a disability prior to enrollment 
and divide it by the actual enrollment (this will provide you 
with a % of children identified with a disability prior to 
enrollment for each program year from 2001-2004): 

2001 - 846/28157 = .030 = 3% 

2002 - 927/30498 = .030 = 3% 

2003 - 1059/33188 = .031 = 3.1% 

2004 - 1099/33023 = .033 = 3.3% 


(The percentage is a proportion that will tell us what 
fraction of the total number of children identified with a 
disability prior to enrollment.) 


Add the percent of children identified with a disability 
prior to enrollment from the years available to you (in this 
example the years are 2001, 2002, 2003, 2004): 

= (2001 + 2002 + 2003 + 2004) = (.03 + .03 + .031 + .033) = 
.124 

Divide the total by the number of years being analyzed: 

= .124/4 = .031 or 3.1% 


(By adding the 4 years worth of data you are determining 
the average which will be used to make an estimate of the % 
of children that will be enrolled in the program with a prior 
disability for the upcoming year.) 


In order to determine the estimate you would need to add 
the actual enrollment from 2001-2004 and calculate the 
average actual enrollment: 

= 28,157 + 30,498 + 33,188 + 33,023 = 124,866 / 4 = 31,217 

And then take the average actual enrollment and multiple it 
by the % calculated above: 

= 31,217 X 3.1% or 31,217 X .031 = 968 
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Disabilities (continued) 


Process Components 

Content Information 

Interpretation 

It is estimated that 3.1 % or 968 children enrolled in HS 
Region “XX” for 2005 will be identified with a disability 
prior to enrollment. 

Based on the estimate HS Region “XX” will not enroll 10% 
of children with disabilities prior to the beginning of the 
program. 


Permission granted to copy for non-commercial uses only. 


59 


Estimating the Number of Eligible Migrant and Seasonal Head 
Start Children and Families 


Process Components 

Purpose 

Question 

Data and Data 
Analysis 


Content Information 

To obtain meaningful estimates of the number of eligible 
children and families for the upcoming program year 

What is the estimate of eligible children and families for the 
upcoming year? 

Information needed = 

1. Actual enrollment (min. of 2 years) 

2. Total number of families served by program 

Grower estimates of eligible families for following program 
year 

Table 1 


Actual Enrollment 


2004 

2005 

Center 1 

275 

265 

Center 2 

112 

125 

Center 3 

213 

215 

Center 4 

75 

80 

Center 5 

95 

95 

Center 6 

300 

295 

Center 7 

266 

270 

Center 8 

100 

101 

Center 9 

125 

125 

Center 10 

275 

270 

Total 

1836 

1841 
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Estimating the Number of Eligible Migrant and Seasonal Head 
Start Children and Families (continued) 

Process Components Content Information 


Data and 
Data Analysis 
continued 


Take the totals in Table 1 and determine the average of the 
two years: 

Table 1 Average = (Total 2004 + Total 2005)/2 
= (1836 + 1841)/2 
= (3677)/2 
= 1838.5 


Table 2 


PIR QUESTION: Total Number of HS or EHS Families 

Served 


2004 

2005 

Number of Families with Children 
in Program 

1080 

1083 

Total 

1080 

1083 


Take the totals in Table 2 and determine the average: 
Table 2 Average = (Total 2004 + Total 2005)/2 
= (1080 + 1083)/2 
= (2163)/2 
= 1081.5 
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Estimating the Number of Eligible Migrant and Seasonal Head 
Start Children and Families (continued) 

Process Components Content Information 


Data and 
Data Analysis 
continued 


Table 3 


Grower Estimates of Eligible Families for 2006 
Program Year 


Low Estimate 

High Estimate 

Center 1 

164 

167 

Center 2 

64 

68 

Center 3 

120 

127 

Center 4 

40 

46 

Center 5 

53 

58 

Center 6 

170 

180 

Center 7 

150 

160 

Center 8 

50 

62 

Center 9 

70 

77 

Center 10 

160 

165 

Total 

1041 

1110 


Take the totals in Table 3 and determine the average of the 
low and high estimate: 

Table 3 Average = (Total Low Estimate + Total High 
Estimate) /2 

= (1041+1 1 10)/2 
= (2151)/2 
= 1075.5 
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Estimating the Number of Eligible Migrant and Seasonal Head 
Start Children and Families (continued) 


Process Components 

Content Information 

Data and 
Data Analysis 
continued 

Determining the Actual Number of Children per Family: 

Take Table 1 Average and Divide by Table 2 Average: 

(Table 1 Average) / (Table 2 Average) = Actual Number of 
Children per Family 

= (1838.5) / (1081.5) 
= 1.69 

Determining the Estimate Number of Children for 
Your Program for 2006 Using Grower Family Estimates: 

Take Table 3 Average and Multiple by the Actual Number 
of Children per Family: 

Estimate Number of Children for Your Program for 2006 
Using Grower Family Estimates = (1075.5) X (1.69) 

= 1817.6 
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Estimating the Number of Eligible Migrant and Seasonal Head 
Start Children and Families (continued) 

Process Components Content Information 


Data and 
Data Analysis 
continued 


Table 4 


Grower Estimates of Eligible Families for 2006 
Program Year 


Low 

Estimate 

High 

Estimate 

Average Grower 
Family Estimate 

Center 1 

164 

167 

331/2=165.5 

Center 2 

64 

68 

132/2=66 

Center 3 

120 

127 

247/2=123.5 

Center 4 

40 

46 

86/2=43 

Center 5 

53 

58 

111/2=55.5 

Center 6 

170 

180 

350/2=175 

Center 7 

150 

160 

310/2=155 

Center 8 

50 

62 

112/2=56 

Center 9 

70 

77 

147/2=73.5 

Center 10 

160 

165 

325/2=162.5 
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Estimating the Number of Eligible Migrant and Seasonal Head 
Start Children and Families (continued) 

Process Components Content Information 


Data and 
Data Analysis 
continued 


Table 5 


Grower Estimates of Eligible Children for 2006 
Program Year by Center 


Column 
A: Average 
Grower 
Family 
Estimate 

Column B: 
Estimated 
Average 
Number of 
Children 
Per Family 

Column C: 
Mulitple 
Column A 
by Column 
B 

Total from 
Column C 

Center 1 

331/2=165.5 

1.69 

(165.5) X 
1.69 

280 

Center 2 

132/2=66 

1.69 

(66) X 1.69 

112 

Center 3 

247/2=123.5 

1.69 

(123.5) X 
1.69 

209 

Center 4 

86/2=43 

1.69 

(43) X 1.69 

73 

Center 5 

111/2=55.5 

1.69 

(55.5) X 
1.69 

94 

Center 6 

350/2=175 

1.69 

(175) X 

1.69 

296 

Center 7 

310/2=155 

1.69 

(155) X 
1.69 

262 

Center 8 

112/2=56 

1.69 

(56) X 1.69 

95 

Center 9 

147/2=73.5 

1.69 

(73.5) X 

1.69 

124 

Center 

10 

325/2=162.5 

1.69 

(162.5) X 
1.69 

275 
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Estimating the Number of Eligible Migrant and Seasonal Head 
Start Children and Families (continued) 

Process Components Content Information 


Data and 
Data Analysis 
continued 


Interpretation 


Table 6 


Actual Enrollment for 2004 and 2005 and Estimated 
Enrollment for 2006 



Actual 

Enrollment 

2004 

Actual 

Enrollment 

2005 

Estimated 

Enrollment 

2006 

Center 1 

275 

265 

280 

Center 2 

112 

125 

112 

Center 3 

213 

215 

209 

Center 4 

75 

80 

73 

Center 5 

95 

95 

94 

Center 6 

300 

295 

296 

Center 7 

266 

270 

262 

Center 8 

100 

101 

95 

Center 9 

125 

125 

124 

Center 10 

275 

270 

275 

Total 

1836 

1841 

1820 


• In this example the estimated number of children for the 
2006 program year is 1820. 

• In this example the estimated number of families for the 
2006 program year is 1076. 

• Things to Consider: 

• Survey a large number of growers. 

• Ask for a range of estimates. 

• Compare with actual data from prior years. 

• Make sure that you have the most accurate data sources. 
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Estimating the Number of Eligible Migrant and Seasonal Head 
Start Children and Families (continued) 

Process Components Content Information 


Interpretation 

continued 


Another estimation procedure that is used in estimating 
migrant and seasonal farmworkers and could possibly be 
used to estimate the number children for migrant and 
seasonal farmworker families is a “bottom-up” approach for 
estimates. This “bottom-up” approach was used by Dr. Alice 
Larson and a group of consultants to estimate the number 
of migrant and seasonal farmworkers for an enumeration 
profiles study conducted for the Bureau of Primary Health 
Care. The enumeration profiles for Arkansas, California, 
Florida, Louisiana, Maryland, North Carolina, Mississippi, 
Oklahoma, Texas, and Washington can be found at: 

http://bphc.hrsa.gov/migrant/Enumeration/ 

EnumerationStudy.htm 
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Estimating the Number of Eligible Migrant and Seasonal Head 
Start Children and Families (continued) 

Process Components Content Information 


Interpretation 

continued 


The following estimate is described within the enumeration 
profiles study: 

• The held agriculture estimate used a “demand for 
labor” (DFL) process that examines the number of 
workers needed to perform temporary agricultural tasks, 
primarily harvesting. The results estimate full-time 
equivalent (FTE) workers required for the task during 
the period of peak labor demand. Calculations, prepared 
for each county, are derived through a formula using 
four elements: 

• DFL = A x H/W x S 
Where: 

A = crop acreage. 

H = hours needed to perform a specific task (e.g., harvest) 

on 

one acre of the crop. 

W = work hours per farmworker per day during 

maximum activity. 

S = season length for peak work activity. 

In order to relate the “demand for labor” calculation to 
MSHS and determine the estimate of eligible MSHS children 
you will need take the “DFL” in each county and multiply by 
the average number of children per family in your program 
to determine the estimate. 
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Infant/Toddler Languages: Program Policies 

Process Components Content Information 


Purpose 

Questions 


Data Collection 


To identify issues and concerns from staff and parents related 
to infant/toddler languages and program policies. 

What concerns do programs have about infan t/toddler 
languages? 

What preferences do parents express related to language use 
in infant/toddler classrooms? 

Sample Qualitative Data - Partial 
Transcript from Questionnaire 
Distributed to MSHS Programs 

Questionnaire item #1: Languages 

Our teachers are primarily monolingual English speakers 
and our teacher assistants are bilingual English and 
Spanish speakers. Ninety percent of the infants and 
toddlers have parents who are monolingual Spanish 
speakers. 

The families of the infants and toddlers in our program 
speak Spanish only at home. Do we need to use English in 
our program or can we do the whole program in Spanish? 

Our program has families that are speak Spanish only 
(37%) and those that speak Spanish and English (63%). 

The children who understand Spanish are spoken to in 
Spanish and those that understand English are spoken to 
in English. Is this okay for our infants and toddlers or do 
we need to worry about teaching them all in both English 
and Spanish? 

There is a concern that children may be confused when 
children are learning at an early age to speak two 
languages that it may delay their primary language 
development. Do infants and toddlers exposed to more 
than one language have language delays? 
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Infant/Toddler Languages: Program Policies (continued) 


Process Components 

Data Collection 
continued 


Content Information 

In infants and toddler’s classrooms it is very important to 
maintain the home language, to help the children develop 
their skills; also they needs to be exposed to common 
English words. 

Our infants and toddlers who are exposed to more than 
one language seem to prosper and become bilingual. 

When I go into a toddler classroom the children will speak 
Spanish to their Spanish speaking friends, English to their 
English speaking friends and depending on the teacher, 
speak in the teacher’s primary language. 

Teachers are encouraged to develop relationships with 
parents, and learn about their interactions with their 
infant/toddler, as far as communication styles. Teachers 
speak with children at every occasion, they describe 
objects and events to them, in both languages, as much 
as possible. Teachers use songs and finger plays that 
encourage language, and praise all efforts by children to 
speak, in whichever language they chose. Teachers are 
instructed to use their primary language with the children, 
unless they are highly fluent in the second language ...it 
is not best practice to have infant/ toddler teachers use 
improper grammar, inappropriate words, or just poorly 
engage and communicate with their young learners. 
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Infant/Toddler Languages: Program Policies (continued) 


Process Components Content Information 


Data Collection 
continued 


Questionnaire item #2: 
Preferences of Parents 


There is a concern about the children that are speaking 
only Spanish that they will struggle as they get to the 
public school system. 

Our parents want home language in the program. That is 
what we do. 

Our parents want both languages in the program. 

Our parents do not want the use of English to be done to 
the extent that they loose Spanish. How do we do we figure 
out how to do this? 

Parents often state a preference for their infants and 
toddlers to learn English. How can we assure them that 
they will do so even if we use Spanish with them. 

Parents prefer a combination of home language and 
English in the classroom, because they think it is important 
for the child to start learning the country’s language, but 
also to keep their roots and culture. 
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Infant/Toddler Languages: Program Policies (continued) 

Process Components Content Information 


Data Analysis 


The first step in analyzing qualitative data is to arrange the 
data in a maneagable format that will facilitate the process of 
assigning codes and themes to sections of the transcript. 

Managing and Analyzing Narrative Data by Developing 
Codes and Themes 


Parents Preferences for Language Use in Infant/Toddler 
Classrooms: 

Text 

Code 

Theme 

Our parents 
want home 
language in the 
program 

Preference for 
Spanish 

Desire to preserve 
home language; 
classrooms 
should support 
child’s first 
language. 

Parents often 
state a preference 
for their infants 
and toddlers to 
learn English. 

Preference for 
English 

Desire for 
children to 
learn English; 
classrooms 
should support 
English 
acquisition. 

Parents prefer a 
combination of 
home language 
and English in 
the classroom 

Our parents 
want both 
languages in the 
program 

Preference for 
both Spanish 
& English 

Desire for 
dual language 
classrooms; 
support 

learning in both 
languages. 
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Infant/Toddler Languages: Program Policies (continued) 

Process Components Content Information 


Interpretation 


Children’s Languages 

Although many infants and toddlers in MSHS programs are 
predominantly from Spanish-speaking families, there appear 
to be important differences in the extent to which children 
are also exposed to English. In many cases, it appears that 
infants and toddlers are truly monolingual - their exposure 
to language is entirely in Spanish. In other cases, however, 
children are clearly exposed to varying amounts of English 
in addition to Spanish. Although the questionnaire did not 
obtain data on the sources, amounts of types of exposure 
across languages, responses from some programs referred to 
toddlers who were equally capable of using both Spanish and 
English. These children were reported to be able to address 
Spanish- and English-speakers in their own language. In 
addition, it appears that the developmental progression of 
both of the children’s languages were relatively parallel (i.e., 
that children were using age- appropriate expressions in both 
of their languages). 

A second important issue identified from the questionnaire 
responses was the language backgrounds of MSHS teaching 
staff. Here, fundamental difference emerged between 
programs regarding the languages of their teachers. Most 
programs appear to staff their infant/toddler classrooms 
with Spanish-speaking teachers; some doing so exclusively, 
based upon an explicit philosophy and/or set of program 
policies. In other cases, program staff are a mix of Spanish- 
and English-monolinguals as well as teachers who speak 
both languages. In a few programs, teachers are primarily 
English monolinguals. 
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Infant/Toddler Languages: Program Policies (continued) 

Process Components Content Information 


Interpretation 

continued 


Children’s Languages continued 

These basic differences appear to exist for a variety of 
reasons: in some cases, programs seem to choose the type(s) 
of languages they want in their teachers; in other cases, 
programs seem to be making do with the available workforce 
of their rural communities. 

Parent Preferences for Language(s) in the Classroom 

Information obtained on parents’ expressed preferences 
for language use in MSHS infant/toddler classrooms varied 
as much as the data on teachers’ language backgrounds. 
Basically, there was evidence for all possible positions on the 
issue. 

Some programs reported that parents want their home 
language (Spanish) to be the language of their child’s 
classroom. Other programs reported that parents often 
state a preference for their child to learn English - even to 
the exclusion of Spanish. Finally, other programs indicated 
that parents express a preference for a combination of 
both languages in the classroom. In some cases, this latter 
perspective was modified by the consideration that exposure 
to English should not be done to the extent that children 
lose their first language. Given the shared governance 
mandate of Head Start programs, and the diversity of parent 
preferences, these considerations are especially important for 
local program practices and policies. 
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Family Strengths and Needs 


Process Components 

Purpose 

Questions 
Data Collection 


Content Information 

To identify strengths and needs of migrant farmworker 
families as they move from one geographical area to another. 

• What needs do families have as they enter the program? 

• What is the impact on individuals and family systems? 

Sample Qualitative Data - Partial Transcript from a 2002 
Parent Interview Conducted in a Migrant and Seasonal Head 
Start Program 

(Family One - husband) 

OK, well, me first, before anything, I came here because 
of the poverty, do you understand me? And for a good 
quality of life, in my country not so much thinking about 
myself but about my smaller sisters so that they can have 
a better education considering that I did not have any, 
and another reason was in my country there are no good 
jobs, and very little work. They pay you very little and 
you never leave the poverty and another reason is that my 
father and I had two brothers that were already here and 
I thought that the more of us that are here the better that 
everybody could help each other out and it would be easier 
to get ahead with our younger sisters. 

(Family One - wife) 

The same to help my parents I came and I would send 
money and then my brother came and then the same 
(interrupted by the husband) 
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Family Strengths and Needs (continued) 


Process Components 

Data Collection 
continued 


Content Information 

(Family One - husband) 

Because in her case, she or better yet sometimes us (in our 
situation) because we were the eldest, we were men but 
in her case the eldest are women, and the ones that were 
going to work, theoretically to help the parents the most 
were the youngest. 


(Family One-wife) 

My dad was already here when I came, he was here, mi 
mother was in Mexico, my dad stayed here for some time 
and than he went to Mexico. 


(Family One-husband) 

More than anything, my parents for example they are 
that type of people that support you - no they never stop, 
for example if your dreams do not become reality you try 
anyway, you know who you are, and they always let us 
because we wanted better than things that they showed us, 
and that was good - they always tried to better us. 


(Family One-husband) 

When I arrived here it was real real rare to see Mexicans, 
Columbians, there was very little people here, the little 
words that I have learned, do you understand me, I 
learned from a book, the principal thing is asking for a job 
in English I repeated and repeated the couple of words, 
when you get to a restaurant and an American comes and 
tells you, he would ask you what are you looking for, and 
than that word I knew and for example, I’m looking for 
job. I had to know and learn. 
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Family Strengths and Needs (continued) 

Process Components Content Information 


Data Analysis 


Again the first step in analyzing qualitative date is to 
arrange the data in a manageable format that will facilitate 
the process of assigning codes and themes to sections of 
the transcript: 


Text 

Code 

Theme 

OK, well, me first, 
before anything, 1 
came here because 
of the poverty, do 
you understand 
me? 

Poverty 

Poverty as a 
motivator for 
migration 

And for a good 
quality of life, in 
my country not 
so much thinking 
about myself but 
about my smaller 
sisters so that 
they can have a 
better education 
considering that 1 
did not have any, 

Family 

opportunity/ 

education 

Increased 
educational 
opportunities for 
family members 

and another reason 
was in my country 
there are no good 
jobs, and very little 
work. 

Work 

Employment 
as a motivator 
migration 

They pay you very 
little and you never 
leave the poverty 

Poverty 

Poverty as a 
motivator for 
migration 
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Family Strengths and Needs (continued) 

Process Components Content Information 


Data Analysis 
continued 


Text (continued) 

Code (continued) 

Theme (continued) 

and another reason 
is that my father and 
I had two brothers 
that were already 
here and I thought 
that the more of 
us that are here the 
better that everybody 
could help each 
other out and it 
would be easier to 
get ahead with our 
younger sisters. 

Support 

Family support 
for economic 
advancement 

(Family One - wife) 
The same to help my 
parents I came and 
I would send money 
and then my brother 
came and then the 
same (interrupted by 
the husband) 

Family 

opportunity/ 

economic 

Family support 
for economic 
advancement 
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Family Strengths and Needs (continued) 

Process Components Content Information 


Data Analysis 
continued 


Text (continued) 

Code (continued) 

Theme (continued) 

(Family One 

Roles/ 

Family structures 

- husband) 

responsibilities 

that have an 

Because in her case, 
she or better yet 
sometimes us (in our 
situation) because 
we were the eldest, 
we were men but in 
her case the eldest 
are women, and the 
ones that were going 
to work, theoretically 
to help the parents 
the most were the 
youngest. 


economic impact 

(Family One-wife) 
My dad was already 
here when I came, he 
was here, my mother 
was in Mexico, my 
dad stayed here for 
some time and than 
he went to Mexico. 

Separation 

Family separation 
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Family Strengths and Needs (continued) 

Process Components Content Information 


Data Analysis 
continued 


Text (continued) 

Code (continued) 

Theme (continued) 

(Family One- 
husband) 

More than anything, 
my parents for 
example they are 
that type of people 
that support you 

- no they never 
stop, for example 
if your dreams do 
not become reality 
you try anyway, you 
know who you are, 
and they always let us 
because we wanted 
better than things 
that they showed us, 
and that was good 

- they always tried to 
better us. 

Support 

Family support 
for economic 
advancement 

(Family One- 
husband) 

When I arrived here 
it was real real rare 
to see Mexicans, 
Columbians, there 
was very little people 
here, 

Community 

Emergent 

community 
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Family Strengths and Needs (continued) 

Process Components Content Information 


Data Analysis 
continued 


Text (continued) 

Code (continued) 

Theme (continued) 

the little words that 
I have learned, do 
you understand me, I 
learned from a book, 
the principal thing 
is asking for a job in 
English I repeated 
and repeated the 
couple of words, 
when you get to a 
restaurant and an 
American comes and 
tells you, he would 
ask you what are you 
looking for, and than 
that word I knew 
and for example, I’m 
looking for job. I had 
to know and learn. 

Language/work 

Impact of 
language on 
employment 
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Family Strengths and Needs (continued) 

Process Components Content Information 


Data Analysis 
continued 


Theme 

Frequency 

Family support for economic 
advancement 

3 

Poverty as a motivator for migration 

2 

Increased educational opportunities for 
family members 

1 

Employment as a motivator migration 

1 

Family structures that have an economic 
impact 

1 

Family separation 

1 

Emergent community 

1 

Impact of language on employment 

1 


Once the theme has been established, count the number 
of times that the theme was identified. At this point, the 
themes can be arranged in order of frequency (the number 
of times that the theme was identified): 
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Family Strengths and Needs (continued) 

Process Components Content Information 


Interpretation 


• Looking at the transcript sample we notice that 
the family left their country of origin for economic 
advancement facilitated by the support of their families 
both in the United States and in their country of origin. 
In turn we notice a restructuring of the family system 
that is divided geographically in which some members 
are in the country of origin and some in the new 
community. From a mental health standpoint this is 
something to note for further examination. 

• Poverty is another indicator that impacted the family’s 
decision to migrate. It is important to note that the 
family’s discussion of poverty is not related to their 
current situation but rather to what prompted them to 
move. For further examination it would be important to 
probe families on their current situation regarding their 
notion of poverty and their current status. 
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Appendix A: Foundations for Data 
Analysis Applications in MSHS 


The Federal Framework for data analysis applications 
includes: 

• Head Start Act, 1998 

• Information Memorandum 

• ACYF-IM-HS-00-03 

• Information Memorandum 

• ACYF-IM-HS-00- 1 8 

• GAO Report, 1992 

Here, we have not sought to interpret the information, only to provide excerpts of some 
of the most important information. Program staff are encouraged to obtain the complete 
versions of these documents for more careful review. 
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Excerpts: The Head Start Act, 1998 

As Amended October 27, 1998 

Quality Standards; Monitoring of Head Start Agencies and Programs 
Sec. 641A [42 U.S.C. 9836A] 

The Secretary shall establish by regulation standards, including minimum levels of overall 
accomplishment, applicable to Head Start agencies, programs, and projects under this 
subchapter, including — additional education performance standards to ensure that the 
children participating in the program, at a minimum: 

• Develop phonemic, print, and numeracy awareness; 

• Understand and use language to communicate for various purposes; 

• Understand and use increasingly complex and varied vocabulary; 

• Develop and demonstrate an appreciation of books; and 

• In the case of non-English background children, progress toward an acquisition of the 
English language. 
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Excerpts: Information Memorandum 
ACYF-I M-HS-00-03 (1/31/2000) 

Subject: 

Initial Guidance on New Legislative Provisions on Performance Standards, Performance 
Measures, Program Self Self-Assessment and Program Monitoring 

The purpose of this Memorandum is to inform grantees about the legislative changes 
on program outcomes, accountability and program improvement in the 1998 Head 
Start Reauthorization. These requirements, applicable to all Head Start grantees serving 
three to five year old children, build on and expand our current efforts to improve and 
document program quality and effectiveness. 


Program Self-Assessment 

Legislative Provision: Section 641A(b)(2)(B) requires that results-based performance 
measures “be adaptable for use in self assessment, peer review, and program evaluation of 
individual Head Start agencies and programs.” 

This new legislative mandate calls for Head Start grantees and delegate agencies to 
augment their current self-assessment efforts by creating a system to track patterns 
and program outcomes and to use this information to inform program planning and 
improvement efforts. 


Program Monitoring 

The new legislative provisions will require that program monitoring incorporates 
examination of how grantees are tracking and making use of data on child and program 
outcomes, as well as patterns of those outcomes. 
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Excerpts: Information Memorandum 
ACYF-I M-HS-00-18 (8/10/2000) 

Subject: 

Using Child Outcomes in Program Self-Assessment 

The purpose of this Memorandum is to inform Head Start grantees and delegate agencies 
about: 

• Steps for incorporating data on child outcomes in their program’s self-assessment. 

• Recommendations for local grantee planning to implement these steps. 

Early Head Start and Migrant Head Start programs are responsible for implementing 
Program Performance Standards, including initial screening and ongoing assessment 
of all children as annual program self-assessment. Grantees should focus on the goal 
of enhancing program self-assessment and continuous program improvement by 
incorporating data on child outcomes. 

The objectives of this initiative are to: 

Improve the content, quality, consistency and credibility of ongoing assessment of 
children. 

Design an approach to analyze data on children’s progress and accomplishments. 

Incorporate child outcome data into program self-assessment and continuous program 
involvement. 

Grantees should ensure that their system for ongoing assessment of children includes 
collection of some data in each of the eight domains of children’s learning and 
development: 

• Language Development 

• Literacy 

• Mathematics 

• Science 

• Creative Arts 

• Social and Emotional Development 
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• Approaches Toward Learning 

• Physical Health and Development 

Grantees must develop a system to analyze data on child outcomes that centers on 
patterns of progress for groups of children over time as they receive services through the 
program year. 

At a minimum, data analysis should compare progress beginning when children enter 
Head Start, at a mid-point in the program year, and when they when they complete the 
program year. 

Data analysis could also be designed to answer questions such as the following: 

• What are the patterns of progress and accomplishments for groups of children in 
different domains and indicators of learning and development? 

• What are the patterns of outcomes for children in different program options, forms of 
service, and service areas? 

• What are the patterns of outcome data for different groups of children, such as (a) 
those that begin Head Start at different levels of functioning and (b) children that 
receive one or two years of Head Start services? 

• What are the trends in outcome data from year to year, in terms of stability and 
change in patterns of progress and levels of accomplishment? 

In most programs, analysis of child outcomes should be based on data from all children 
enrolled. Grantees that serve very large numbers of children may consider, with the 
assistance of appropriate research or evaluation experts, approaches to analyzing outcome 
data from a sample of children selected to be representative of the full population of 
children being served. 

Data analysis on patterns of child outcomes should be incorporated into the overall self- 
assessment system and in reporting to agency management teams, Policy Committees and 
Policy Councils and governing bodies. 
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Excerpts: Quantitative Data Analysis: An Introduction 
Report to Program Evaluation and Methodology Division. 
United States General Accounting Office, 1992 

Successful data analysis, whether quantitative or qualitative, requires: 

• Understanding of a variety of data analysis methods, 

• Planning data analysis early in a project and making revisions in the plan as the work 
develops; 

• Understanding which methods will best answer the study questions posed, given the 
data that have been collected; 

• Once the analysis is finished recognizing how weaknesses in the data or the analysis 
affect the conclusions that can properly be drawn. 

Data Analysis 

After the data are collected, evaluators need to see whether their expectations regarding 
data characteristics and quality have been met. Choice among possible analyses should 
be based partly on the nature of the data — for example, whether many observed values 
are small and a few are large and whether the data are complete. If the data do not fit the 
assumptions of the methods they had planned to use, the evaluators have to regroup and 
decide what to do with the data they have. 

Balancing the analytic alternatives calls for the exercise of considerable judgment. 


Generic Types of Quantitative Questions 


Generic Question 

What is a typical value of the variable? 

How much spread is there among the cases? To what extent are two 
or more variables associated? 


To what extent are there causal relationships among two or more 
variables? 
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Appendix B: Using Excel To Help 
Analyze Your Quantitative Data 


The task of reviewing and analyzing your quantitative data is made easier with a computer 
software spreadsheet. One such example is Microsoft’s Excel. You will want to begin the 
process by considering the questions that you want to address. Some of the questions are 
close-ended questions (e.g., Where are you living now? Where did you live before?) that 
can be answered using a limited number of responses (e.g., selection 1, 2, 3, or 4; yes/no; 
etc.). 

For example, if you were to utilize the following questions (i.e., 

• Where are you living now? 

• How many children (under age 5) live with you? 

• How old are each of the children? 

• How many children do you have in Migrant and Seasonal Head Start?) 



and put them into a spreadsheet, it might look something like the following: 


Since the first question (Where are you living?) is not a numerical response, you might 
want to create some geographical categories (e.g., East, South, West, North) or names of 
farms or camps (e.g., Campo Verde, Campo Chico, etc.) to which you could then enter 
a “1” for each parent responding. For questions like How old are each of the children?, 
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you may want to assign age groups to separate columns. After entering information into 
the spreadsheet, you can quickly sum up the data in the columns by using the Auto Sum 
button. To double-check that the information in the selected cell is correct, look at the 
function (fx) — in this case, the cell J14 has the following function: J14 = SUM (J3:J13) 

— that is, all the cells from J3 through J13 have been summed together.. .and the total 
equals seventeen. 

Using a spreadsheet will also allow you to create other functions, such as percentages. In 
the example below, the responses from ten parent questionnaires have been summed, then 
percentages have been calculated by creating functions. For example, the cell J16 has the 
function (fx = J14/F14) — that is, the total number of children in the Migrant and Seasonal 
Flead Start program (in cell J 14) is divided by the total number of children “Age<5” (in 
cell F14). 15/19 = 0.7894 and, if you click on the “Percent Style” button, you can change 

this notation to “79%” (it automatically rounds up). 







With a computer spreadsheet, you can also create a summary table (see below) 




Highlighting the information that you would like to display, you can then go to the 
Standard Toolbar and select the Chart Wizard to help you create a graphic to represent 
your information. v 
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Step by step instructions will then walk you through the process (see below). 



From a number of different options (i.e., bar chart, line graph, pie chart), select the one 
that you think will best represent your data to your target audience (e.g., Policy Council, 
Management Staff) in an easy-to-understand manner. You can try different options, share 
the results with your “team,” and select one that best suits your needs. 
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You can then print it out, share it with your target audience to get their feedback, and cut 
and paste the finished product into your report, as needed. 
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Appendix C: Supplemental 

Resources 
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Y N 

Y 

11 O «/ 

31 O' O 

s i or a 

7.1 O 

non 

yidp 

5.2 0 

7.20 


3.3170 

5.3 0/0 

5.4 O' 0 

7.3 0 


NNA 


A. Subscale (Items 38-43) Score _ 

B. Number of items scored: ( t / 


PARENTS & STAFF Average Score (A + B) 

Space & Furnishings 
Personal Care 

Total and Average Scores 
Total Score # of lisa» StQIsd 

H 7 if 

Average Score 

1* 

Tii 4 

Language-Reasoning 

Activities 


71 1 O 0 

a 7 it> 

U. 70 

Interaction 

jt | 

^ i 

Program Structure 
Parents & Staff 

1*2 If 

V - S' 

7m L 


TOTAL 

5o\ 

M .(o') 


58 
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101 


ECERS-R Profile 


Gocet/Sdiooi: 

TeacherfiVOiasroom - 


Obwnwion t: 

Ob+ctruton 2: — _ I -fJ-j- y- 


Obacner 

Obrerrer 


I. Space & Furnishings 
(I -8) 


Qb*i Qb*. 3 

53 □ 

•*ai|c ubiulc 


II. Personal Care Routines 
(9-14) 


[*E □ 



111 Language* Reasoning 

05-19 


he n 


IV Activities (19-28) 

E3 □ 


V. Interaction (29-33) 

OlE] □ 








IS ftooki aivi pictures 









i 

— 




It. Informal use of language 



19. Ft*e motor 
20 An 

21. Musi o' movement 

22. Blocks 

23. Sand/water 

24 Dramatic play 

25 Ntcurt/socnce 

26 Math/number 

27. Use of TV, video, and/cc computers 
26 Promoting acceptance of diversity 



29 Supervision of gross motor activities 

30 General wpervition of children 

31 Discipline 

32 Stiff -child interactions 

33 Interactions among children 


VL Program Structure 
(34 - 37) 

53 n 

VII Parents and Staff 

(38-43) 

53 □ 


Average Subscale Scores 



34 Schedule 

35 Pree play 

36 Group time 

37 Provisions for children with disabilities 

36 Provisions for parents 

39 Piuviiiora for pctutiul needs of stafT 

40 Plroviiioca for profession*] need* of staff 

4 1 St iff interaction and coopcrafsod 

42 Scpermwu and evaluation of staff 

43 Opportunties for professional growth 

SPACE A FURNISHINGS 
POISON AL CARE 
LANGUAGE-REASONING 
ACTIVITIES 
INTERACTION 
PROGRAM STRUCTURE 
PARENTS A STAFF 
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Data Analysis Evaluation Checklist 

(i.e. teacher credentials, enrollment, community assessment, etc) 


Process Components 

Evaluation Questions 

Evidence 

Purpose: 

What Do We Do & Why? 

Was the data analysis process consistent with 
federal requirements and other, relevant 
regulations? 


Questions: 

What Do We Want To Know? 

Were the questions worded in a way that 
was consistent with federal standards, other 
regulations, and organizational purposes? 
Were the questions effective in guiding the 
collection and analysis of data? 


Data Collection: 

What Information Can Help 
Us Answer Our Question(s)? 

How well did the data collection plan work? 
Was there enough time allotted to obtain the 
necessary information? Were data sources 
used that were not effective? Do additional 
data sources exist that were not utilized? Did 
the team collect too little data or too much? 


Data Analysis: 

What Are Our Results? 

Which procedures were chosen? Did these 
conform to the purposes and questions? 
Were there additional procedures that could 
be used in the future? 


Interpretation: 

What Do The Results Mean? 

How well did the interpretation process 
work? What information was used to 
provide a context for the interpretation 
of the results? Was additional relevant 
information not utilized for interpretation? 
Did team members disagree over the 
interpretation of the data or was there 
consensus? 


Writing/Reporting/ 

Disseminating: 

What Do We Have To Say? 
How Do Tell The Story of The 
Data? 

How well did the writing tell the story of the 
data? Did the intended audience find the 
presentation of information effective? 


Evaluation: 

What Did We Learn About 
Our Data Analysis Process? 

Did we evaluate? 
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